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Abstract. Curtin University’s Talking Heads (TH) combine an MPEG-4 
compliant Facial Animation Engine (FAE), a Text To Emotional Speech 
Synthesiser (TTES), and a multi-modal Dialogue Manager (DM), that accesses 
a Knowledge Base (KB) and outputs Virtual Human Markup Language 
(VHML) text which drives the TTES and FAE. A user enters a question and an 
animated TH responds with a believable and affective voice and actions. 
However, this response to the user is normally marked up in VHML by the KB 
developer to produce the required facial gestures and emotional display. A real 
person does not react by fixed rules but on personality, beliefs, good and bad 
previous experiences, and training. This paper reviews personality theories and 
models relevant to THs, and then discusses the research at Curtin over the last 
five years in implementing and evaluating personality models. Finally the paper 
proposes an active, adaptive personality model to unify that work. 


1 Introduction 


Curtin’s THs (or Embodied Conversational Agents (ECAs) [1, 2]), combine an 
MPEG-4 [3, 4] compliant FAE [5], a TTES [6, 7], a multi-modal DM [8-10], and a 
VHML [11-15] KB. A typical application uses a client to display one or more THs 
connected to one or more servers for the TTES and the DM. The use of VHML means 
that the application client can transparently be a plain text interface, a Web page, 
voice only, a TH with complex facial gestures, voice and emotion, or an entire Virtual 
Human with body language. A response from the KB could then simply be displayed, 
spoken, or enacted by a TH. If the client were a TH newsreader, a VHML news item 
example from the KB might be: 


<sad> 

<pitch range="+150%" middle="-10%"> 

And here's the latest <pitch middle= " -18%">news . </pitch> 
<pause 1 ength="short "/> Detectives investigating the 
<emph >brut a l</emph> murder of Sarah Payne, <blink/> have 
received 200 fresh calls from the public. 

</pi tchxsmile/> 

</sad> 


The text from this news report has been marked up by the KB developer, based 
upon their feeling for how the news should be read: overall a sad voice and posture, a 
pause after the first sentence for the listener to pay attention, emphasis on the word 
“brutal”, a short blink at the sentence break, and a smile at the end of the second 
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sentence since it probably indicates good news. However, it must be noted that the 
marking up of this dynamic and constantly changing information by the KB developer 
is very tedious and time consuming. It would be more appropriate for the TH to deliver 
the information with its own emphasis, gestures, and emotions. That is, the TH would 
filter this information through its own personality by marking up the plain text with 
VHML, and this would impart a different “look and sound” to the news. This 
personality filtering is applicable to any information delivered by a TH. Ultimately, the 
TH’s personality would make the interaction more human and more humane. 

2 Personality Theories 

It has been proposed that personality, mood and emotion are necessary for building 
believable EC As [16-21]. Time duration is the main difference between these three 
components [20, 22, 23]. Figure 1 (from [20]) shows that emotion is a momentary 
state and mood is more static than emotion. Personality in contrast, does not change 
over time. Wilson[22] suggested that these three layers have different priorities for 
controlling behaviour: emotions have the highest priority whilst personality has the 
lowest. Moffat [23] also indicated a focus difference between emotion and 
personality. Emotions are focused on specific events, actions and objects whilst 
personality is more general. The investigation of relation and interaction among 
emotion, mood and personality is necessary to build a powerful personality model. 
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Fig. 1 . Personality, Mood and Emotion [20] 


Emotion 

Emotion is a essential part of a believable EC A that communicates with humans [19, 
24, 25]. Computer science researchers concerned with facial, body and vocal 
expressions, as well as the appraisal of an agent’s emotions based on its reactions to 
objects, events and actions, have developed a number of emotional models (Reilly 
and Bates’ Em [26], Gratch’s Emile [27], Velasquez’s Cathexis [25] and Prendinger 
et al. SCREAM [28]). Many of these employed the cognitive appraisal model of OCC 
[29]. Emotions in the OCC model are the result of three kinds of appraisals: appraisal 
of the pleasantness of events with respect to the agent’s goals, of the approval of the 
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Table 1 . Basic emotions of the OCC and Ekman models 


Model 

Emotion types 

OCC 

Happy-for Resentment Gloating Pity Joy Distress Pride Shame 
Admiration Reproach Love Hate Satisfaction Fears-confirmed Relief 
Disappointment Gratification Remorse Gratitude Anger Hope and Fear 

Ekman 

Happiness Anger Disgust Sadness Surprise and Fear 


actions of the agent or another agent with respect to a set of standards for behaviour, 
and of the liking of objects with respect to the attitudes of the agent [26]. The 22 
emotion types modelled in OCC contrasts strongly with the 6 basic emotions of 
Ekman [30] that are widely used for facial and vocal expressions of an EC A. Table 1 
shows the emotion types of these models. 

Bartneck [19], using OCC, split the emotion process into the following phases: 

1. Find what emotional types are affected using the evaluation of an event, action 
or object based on the ECA’s knowledge. 

2. Calculate the intensities of the affected emotional types. 

3. Interact the emotion value with the ECA’s current emotional state. 

4. Map the 22 OCC emotion types to a possible lower number of expression 
emotions available to the ECA. The ECA then renders its emotion through 
facial, body and vocal expressions. 

Additionally, a phase is needed to blend emotion types to one emotion. This phase 
solves the problem of more than one emotion type being generated at each moment. 
For example, the ECA may be both happy because of an interesting topic, and 
disgusted with the user’s silly questions. 

Mood 

Adopting a psychobiological perspective, mood is explained by Velasquez [25] as a 
low level of arousal within an emotional system whilst emotion is explained as a high 
level of arousal. Mood is a necessary bridge which links personality to emotion 
expression since a personality models’ high level descriptions make it difficult to 
directly control the emotions that are visible on the virtual face [20]. 

Kshirsagar [20] simply defined mood as one dimensional, going from bad mood to 
good mood, while Rousseau’s social-psychological model [31] divided mood into two 
categories: agent-oriented moods, and self-oriented moods. The agent-oriented moods 
are directed toward other individuals whereas the self-oriented moods are not[31]. 
The distinction between self-oriented and agent-oriented moods can make a character 
rather happy in general, yet remain angry at a particular individual because of what 
he/she did to the character [31]. 

Personality 

Personality represents those characteristics of the individual that account for consistent 
patterns of thinking, feeling and acting[32]. This broad definition of personality also 
indicates that personality makes people different from one another [33]. The definition 
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can be adopted to describe the personality of an ECA as a relatively stable pattern that 
affects emotion expression and behaviour, and differentiates it from another ECA. 
Since psychologists have been interested in personality since the beginning of the last 
century, many researchers [20, 34, 35] have used the psychology literature for 
modelling an ECA’s personality. 

The first comprehensive theory of personality was Freud’s psychoanalytical theory 
that attributed an individual’s thoughts and actions to the unconscious in regulating 
behaviour [36]. Biological theories attempted to explain differences in behaviour in 
terms of differences in physiology, particularly brain function [33]. Social learning 
theories explained differences in behaviour in terms of a continuous reciprocal 
interaction between cognitive, behavioural, and environmental determinants [37]. 
Gordon Allport[42], Raymond Cattell and Hans Eysenck proposed trait theories that 
used traits as descriptors to describe personality [33]. Traits are labels given to 
consistent aspects of personality and are viewed as continuous dimensions [33]. 
Whether all people possess the same dimensions of personality and differ only in the 
value is still debated by psychologists. However, most trait theorists assume that all 
people have a fixed number of basic dimensions of personality: Eysenck and 
Eysenck’s [38] two basic dimensions of personality, Cattell's 16 dimensions and 
Costa & McCrae’s [39] Five Factor Model and so on. These theories are widely 
recognised, but there are few implemented ECA personality models (see Table 2). 

Table 2. Trait theories 


Trait theories 

Dimensions 

Computer Science Cases 

Eysenck & 

Eysenck's 2 basic 
dimensions 

Extraversion 

Neuroticism 


Five Factor 

Model 

Openness 

Conscientiousness 

Extraversion 

Agreeableness 

Neuroticism 

•Multilayer Personality Model [20] 

•Fuzzy Agents with Personality [41] 

•Andre’s three lifelike characters project [18] 

Interpersonal 

theory 

Dominance 

Friendliness 

•Breese and Ball’s emotion/personality model [17] 
•Nass et al. computer-based personality [34] 


Most of the computational models of personality are based on trait theories because 
the traits that originate in everyday language are easy to understand, and the 
conversion from trait dimensions to a computational model is simple. 

Rousseau [35] proposed 16 dimensions of personality that are based on the 
processes that intelligent agents usually perform. These 16 dimensions have been 
successfully applied in Cybercafe and Bui’s ParleE [43]. The modelling of ECA’s 
personality is not limited to trait theories. For example, the personality model of 
Rousseau and Hayes-Roth [31] adopted the psychology theories of social learning. 

Psychologists Byrne et al. [44] indicated that humans prefer to interact with others 
who are similar in personality to themselves. Following this finding, human computer 
interaction research by Nass et al. [34] found that users are more satisfied with the 
interaction when the user and the computer have similar personalities. For example, a 
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submissive user prefers a submissive EC A instead of a dominant one. It follows that 
an ECA system that allows a user to choose or develop a suitable personality for the 
ECA they interact with will result in a more satisfying interaction. 


3 Curtin’s Implemented Personality Applications 

Shepherdson [45] focused on specific personality traits that could be modelled in an 
MPEG-4 compliant TH: dominance, submissiveness, friendliness and unfriendliness 
which are of their importance in interpersonal communication. The study used facial 
expressions of emotions, eye behaviour (gaze behaviour, eyelid closure and openness, 
eyelid blinks), mouth gestures, head movements (head-turning, head-nodding) and 
gestures (eyebrow raising and lowering) to convey different personality traits. Each of 
the chosen personality traits was modelled and stored in a personality (PST) file that 
directed the display and control of facial expressions. 

Shepherdson [45] had the three hypotheses - the TH implementing personality will: 

1. be able to communicate the spoken information to the user more effectively, 

2. correctly exhibit personalities as intended by the author, 

3. be perceived by users as a more humane interface. 

Only the second hypothesis was proven conclusively although anecdotal evidence 
indicated some support for the other two. Also, to better differentiate the 
personalities, a bigger set of gestures is required and to be an effective 
communicator, a TH requires clear and audible speech from a better speech 
synthesis. Further, the research concluded that since personality is not just a single 
value (e.g. dominant), but a continuum of values, future work should allow the 
blending of personality traits. 

Gulland [46] evaluated a case study built on the Five Factor Model. The Idolum 
framework demonstrated an idle-time behaviour of moods and emotions controlled by 
a consistent personality. In order to be more believable, Idolum took into account 
aspects of personality, mood and stimuli elements from psychological models such as 
a time cycle (winter/summer), the weather, or a manic/depressive cycle that can effect 
on emotional behaviour (see Figure 2). To avoid predictable and repetitive actions, 
which can hinder the believability of a character [47], it was seen as important to 
incorporate a small random factor in the change of emotional levels and the relevant 
behaviours calculated from these levels. 

Another problem considered by Gulland was the action-expression problem [48] 
where two or more emotions are at similar levels and may compete unnaturally. This 
was solved by tracking the rate of change in each emotion and/or replacing conflicting 
emotions within a certain range of each other, with “confused-type” behaviours. 

Building on this, Dam and de Souza [49] reported on a TH system that “learnt” the 
user’s opinions about weather conditions so as to mimic the user’s likes and dislikes. 
Although this “mimic personality” research was done in a limited domain, the results 
supported the research on personality compatibility done by Nass et al [34]. 
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Effect of a manic-depressive cycle on the strength of 
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Fig. 2. Contrived example of the effect of a manic/depressive cycle on the strength of 
“happiness” exhibited (from Gulland [46]) 

Beard and Reid [50, 51] and Marriott [10] have continued in this research area by 
addressing the issues of personalities in THs, and the issues of human-TH interaction 
concerned with personality. For example, a user may be able to determine the 
personality of a TH through its voice and actions, but the opposite - the TH 
determining the personality of the user - is not true due to the single-modal input to the 
TH system from the user - plain text. To this end, the existing DM has been modified 
to cater for multi-modal input (such as emotion values). Although convoluted, the user 
can now also include emotion in the dialogue with a TH by manipulating graphical 
slider values for emotions. These emotions may enable a TH to determine the 
personality of the user over many interactions. 

4 Curtin’s Current Personality System 

The applications of the previous section did not fully integrate the personality model 
into the TH - it was always an add-on either to the server or to the client. Current 
research efforts are concentrating on developing an integrated framework that can be 
used by both the client and the server, and that enables the displaying and evaluation 
of different personality theories. Figure 3 is a schematic of the developing system. 

The Active Personality Model is Object Oriented code that is loaded and executed 
at runtime on both the server and client. The network channel will allow both system 
and user personality models to be used interchangeably, with the model being either 
sent from the server to the client or vice versa. The ability to create and load client- 
side user personality models allows for transparent research by disparate groups. 

The Active Personality Model will “remember” user interactions through the use of 
per-user files that contain learnt preferences about how to interact with each user. The 
user interaction will change over time as the ECA personality changes the per-user 
file preferences. 

This research is ongoing, with the existing ECA framework currently being 
modified to accommodate the Active Personality Model and its requirement to be 
used on both the client and the server. Several simple but easily identifiable 
personalities are currently being modelled for implementation. Evaluation is planned 
for late 2005. 
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Fig. 3. An Active Personality Model 


5 Conclusion 

The design, implementation and evaluation of a personality model for an ECA is 
difficult, due to the “human” nature of personality. Some initial steps have been 
taken, and the current study, especially the evaluation method and the evaluation 
results, should be beneficial to other researchers in this relatively new area of ECA 
construction. 
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